Improving Naive Bayesian Classifier by Discriminative Training

Discriminative classifiers such as Support Vector Machines (SVM) directly learn a discriminant function or a posterior probability model to perform classification. On the other hand, generative classifiers often learn a joint probability model and then use the Bayes rule to construct a posterior classifier. In general, generative classifiers are not as accurate as discriminative classifiers. However generative classifiers provide a principled way to deal with the missing information problem, which discriminative classifiers cannot easily handle. To achieve good performance in various classification tasks, it is better to combine these two strategies. In this paper, we develop a method to train one of the popular generative classifiers, the Naive Bayesian classifier (NB) in a discriminative way. We name this new model as the Discriminative Naive Bayesian classifier. We provide theoretic justifications, outline the algorithm, and perform a serious of experiments on benchmark real-world datasets to demonstrate our model’s advantages. Its performance outperforms NB in classification tasks and outperforms SVM in handling missing information tasks.